OK, this incorporates all feedback.
@epoberezkin I checked on the things that use schemaArray (items, anyOf, allOf, oneOf) and none of them have a minimum length requirement so I removed that. None specify a default, so I did not add one.

Here are the individual commits (review it like this, NOT as one giant diff):

Formatting standardization, tie hyper-schema's "schemaArray" to validation's:
ef13c49

Move definitions to top but do not change anything else (more standardization):
a8be3fd

Add "$ref" for subschemas:
018fe92

Update for numeric exclusives, empty schema arrays, give string arrays a default of empty array:
983e9d3

@Relequestual notJsonReference has nothing to do with hyper-schema. It implements this part of the core spec:

All other properties in a "$ref" object MUST be ignored.

What that means is that if "$ref" is present, the object cannot be used as a regular JSON Schema. So it should not validate as one.

To walk through it, let's start with the notJsonReference schema:

"notJsonReference": {
    "not": {
        "required": [ "$ref" ]
    }
}

The inner schema, {"required": [ "$ref" ] } passes validation if a "$ref" property is present (or if the instance is not an object). It doesn't matter what the value of "$ref" is, the only thing that matters is whether it is there.

So, when negated, this means that non-objects fail validation (because there is no "type" restriction in the inner schema) and objects with "$ref" fail validation, regardless of the value of "$ref" (because "$ref" being present at all causes the "required" to pass).

Now let's take a look at the oneOf/allOf construct:

"subSchema": {
    "oneOf": [
        {   
            "allOf": [
                { "$ref": "#" },
                { "$ref": "#/definitions/notJsonReference" }
            ]   
        },  
        { "$ref": "#/definitions/jsonReference" }
    ]   
}   

This "oneOf" says that a subschema must either be a regular schema (the "allOf" branch) or a JSON Reference (the "#/definitions/jsonReference" branch) but not both.

So consider this instance:

{
    "type": "object",
    "description": "Lorem ipsum...",
    "$ref": "#/definitions/something"
}

This is both trying to be a regular schema (type and description) and a reference ($ref). Without "notJsonReference", this would validate as both a regular schema and a reference, which is wrong.

Even though there's nothing structural that would otherwise disqualify this as a schema, the spec says that if "$ref" is present, the other fields MUST be ignored. So even if they look like a schema, they will not behave like one. So this instance should not validate as a regular schema. Without "notJsonReference", it would validate as a regular schema because JSON Schema does not forbid unknown properties. So we need to specifcially forbid $ref, which is what "#/definitions/notJsonReference" does.

Instead, it should validate as a reference, because the value of "$ref" is a URI Reference string. And then the other fields are correctly ignored.

Finally, consider

{
    "$ref": 42
}

This should fail validation entirely. It will fail the "jsonReference" branch of the "oneOf" because 42 is not a string (much less a URI Reference). But without "notJsonReference", this would successfully validate as a regular schema, because unknown properties do not cause validation to fail. So we need "notJsonReference" to ensure that this fails the regular schema branch of the oneOf as well as the JSON Reference branch.

With "notJsonReference" involved, this instance correctly fails validation entirely.

To summarize: IF "$ref" is present, the instance must validate as a JSON Reference and not as anything else. If it fails to validate as a JSON Reference, it must fail validation entirely. ELSE ("$ref" is not present) it passes validation if it is otherwise a regular JSON Schema.

@handrews your metaschema is correct and explanation makes perfect sense. Do you see though how difficult it is for everybody to see the intention (even though @Relequestual has seen a couple of schemas) - although your schema achieves the intention you describe, it is very difficult to understand what it does.

Also, I hope you notice the language you use to describe this intention (IF/THEN/ELSE - you even put two of these words in capitals, to make it clearer). This schema would have been very easy for us to understand and wouldn't have raised the question if these keywords actually existed:

{
  "if": { "required": ["$ref"] },
  "then": { "$ref": "#/definitions/jsonReference" },
  "else": { "$ref": "#" }
}

And it doesn't prescribe any order of execution any more than allOf/anyOf/oneOf do. Subschemas in the latter keywords can be evaluated in any order (although all validators I know evaluate them in the order of array items and short-circuit, unless the mode is to collect all possible errors). In the same way subschemas in if/then/else can be evaluated in any order - it will not affect the final result (although I can't see why validators will not do the sensible thing here as well).

This validation scenario is not constrained to meta-schema. It is very common in real world. And every time, without if/then/else you have to resort to the equivalent schema without if/then/else (see #180), as you did in meta-schema. The problem with that approach is that it is very difficult to see the intention behind this equivalent schema.

If you don't want to see three "imperative looking" keywords the same can be expressed as:

{
  "conditional": [
    { "required": ["$ref"] },
    { "$ref": "#/definitions/jsonReference" },
    { "$ref": "#" }
  ]
}

where the keyword "conditional" must have exactly 2 or 3 items ("else" is optional). That's how lispy languages do it (e.g., clojure).

But this way or another we need a way to express it, as you can see from your own explanation.

@epoberezkin Please don't take over PRs with other issues.

@handrews ok. sorry. couldn't resist... Should've put the same comment in the issue.

couldn't resist... Should've put the same comment in the issue.

No worries. I read the issue first anyway and it all worked out as it was.

@handrews This PR seems to introduce a few typos, mostly missing commas.

Can this get broken up into a smaller series of PRs?

There's a few things to note that would make our life easier, like I don't think boolean "exclusiveMinimum" actually depends on "minimum" existing (it just doesn't make any sense by itself).

@awwright it's already broken into smaller commits. Just review the individual commits. If you tell me what you need changed I will apply the changes to each commit as needed. GitHub doesn't do stacked PRs at all easily.

The dependency for the exclusives comes directly from the existing meta-schema and I don't want to change it. "it doesn't make any sense by itself" is exactly what that is trying to show. There's no need to take that out when it was fine before.

Fixed various typos / missing comments /etc.

Since the commit hashes changed, I have updated the links to the individual commits in #168 (comment)

@handrews This does a lot of things, I'm still reviewing and trying to figure out if GitHub is presenting this to me correctly. I pulled 7cee490, what is that?

Maybe I'm on the wrong end of a caching server, GitHub is still showing me the presumably-old version with a missing comma. It's not what I pulled, but it's not ce456a6 either.

It's not just you, I'm not sure what's going on. I will take a look at it.

@awwright @Relequestual I don't know what the problem was but I amended the last commit (reworded the commit message) and force-pushed and it has the right commits now:

Formatting standardization, tie hyper-schema's "schemaArray" to validation's:
ef13c49

Move definitions to top but do not change anything else (more standardization):
a8be3fd

Add "$ref" for subschemas:
018fe92

Update for numeric exclusives, empty schema arrays, give string arrays a default of empty array:
983e9d3

As a reminder, the goal is to match the current specification and not change anything more than that.

• The $ref URIs in hyper-schema are not relative because they (specifically the "allOf") were not relative before. This is not a change.
• The subschema approach is necessary because $ref is not allowed at the root. We are considering changing that but that belongs in Can a schema document just be a "$ref"? #174.
• Yes, exclusiveMinimum/exclusiveMaximum look weird. That is how the spec is currently written- they are either numbers, OR they are booleans and if so require the presence of minimum/maximum respectively (the dependencies usage is what the current meta-schema does). Again, it's weird, but not a change from the spec. Proposed changes such as dropping the boolean form entirely should get their own issue and not block this PR.

Right. Thanks for explaining. That makes sense. None of these are changes, just fixes to bring the meta-schema in line with what the spec actually says. Great stuff. I'm totally happy with this! (Although we should make sure we capture the problems that exist, in new issues.)

Although we should make sure we capture the problems that exist, in new issues.

#182 Add Boolean schemas to meta-schemas
#183 Relative references
The question of what to do about the exclusiveMinimum/Maximum can be continued in #77

I keep reviewing this, but it's just too difficult to tease apart all of the things going on together, and that's holding up progress on this which is unfortunate. So I'll close this out, but we should deal with all of the issues presented here.

I filed #185 to deal with the case of "exclusive*" properties.

For formatting conserns, I'd like to see a particular rationale for why whitespace should be formatted the way it is. We should document these patterns in README.

I noticed anyOf/allOf/oneOf still requires at least one item, so we can't remove "minItems": 1 from the meta-schema quite yet. Let's figure out how to deal with that in a new issue.

There's a lot going on here so I'm probably missing some stuff, I'd like to comment on these individually so please let me know if there was anything else being dealt with here.

I keep reviewing this, but it's just too difficult to tease apart all of the things going on together

Do you not understand how to review individual commits? I would like a yes or no answer on this.

@handrews I'm not even sure what to say. Yes, I see the feature to review commits. But there's multiple distinct issues here, that I would like to be able to merge individually, instead of having to merge everything at-once.

@awwright Then go merge #186 and I will feed you the PRs one by one right now.

@awwright You also could have tried telling me what these "multiple distinct issues" are. With the exception of the point we're arguing in #185 I made every single change you asked for, immediately. I went on IRC since you wouldn't do it here. I walked you through everything repeatedly. I answered questions, and I changed things and updated the stack appropriately.

I do all of that, and you just close it because of unspecified "issues" that you won't describe or give me a chance to fix. Everyone else who reviewed this had no trouble articulating their concerns, which I also addressed. But you just won't.

Even when closing the PR you're incapable of actually articulating the real problems, which we still don't know.

@awwright I was about to split off the exclusive* bits since you opened #185, but you didn't even come back here and ask for that, you just closed it. And even though you commented here after I opened #186 in an effort to accommodate you once you closed this PR, you didn't bother to comment on that at all.

Should be noted that #188 fixes part of this.